Perceptual oddities: assessing the relationship between film editing and prediction processes.

During film viewing, humans parse sequences of individual shots into larger narrative structures, often weaving transitions at edit points into an apparently seamless and continuous flow. Editing helps filmmakers manipulate visual transitions to induce feelings of fluency/disfluency, tension/relief, curiosity, expectation and several emotional responses. We propose that the perceptual dynamics induced by film editing can be captured by a predictive processing (PP) framework. We hypothesise that visual discontinuities at edit points produce discrepancies between anticipated and actual sensory input, leading to prediction error. Further, we propose that the magnitude of prediction error depends on the predictability of each shot within the narrative flow, and lay out an account based on conflict monitoring. We test this hypothesis in two empirical studies measuring electroencephalography (EEG) during passive viewing of film excerpts, as well as behavioural responses during an active edit detection task. We report the neural and behavioural modulations at editing boundaries across three levels of narrative depth, showing greater modulations for edits spanning less predictable, deeper narrative transitions. Overall, our contribution lays the groundwork for understanding film editing from a PP perspective. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectivess'.

Exploring sensory sensitivity, cortical excitability, and habituation in episodic migraine, as a function of age and disease severity, using the pattern-reversal task.

BACKGROUND: Migraine is a cyclic, neurosensory disorder characterized by recurrent headaches and altered sensory processing. The latter is manifested in hypersensitivity to visual stimuli, measured with questionnaires and sensory thresholds, as well as in abnormal cortical excitability and a lack of habituation, assessed with visual evoked potentials elicited by pattern-reversal stimulation. Here, the goal was to determine whether factors such as age and/or disease severity may exert a modulatory influence on sensory sensitivity, cortical excitability, and habituation. METHODS: Two similar experiments were carried out, the first comparing 24 young, episodic migraine patients and 28 healthy age- and gender-matched controls and the second 36 middle-aged, episodic migraine patients and 30 healthy age- and gender-matched controls. A neurologist confirmed the diagnoses. Migraine phases were obtained using eDiaries. Sensory sensitivity was assessed with the Sensory Perception Quotient and group comparisons were carried out. We obtained pattern-reversal visual evoked potentials and calculated the N1-P1 Peak-to-Peak amplitude. Two linear mixed-effects models were fitted to these data. The first model had Block (first block, last block) and Group (patients, controls) as fixed factors, whereas the second model had Trial (all trials) and Group as fixed factors. Participant was included as a random factor in both. N1-P1 first block amplitude was used to assess cortical excitability and habituation was defined as a decrease of N1-P1 amplitude across Blocks/Trials. Both experiments were performed interictally. RESULTS: The final samples consisted of 18 patients with episodic migraine and 27 headache-free controls (first experiment) and 19 patients and 29 controls (second experiment). In both experiments, patients reported increased visual hypersensitivity on the Sensory Perception Quotient as compared to controls. Regarding N1-P1 peak-to-peak data, there was no main effect of Group, indicating no differences in cortical excitability between groups. Finally, significant main effects of both Block and Trial were found indicating habituation in both groups, regardless of age and headache frequency. CONCLUSIONS: The results of this study yielded evidence for significant hypersensitivity in patients but no significant differences in either habituation or cortical excitability, as compared to headache-free controls. Although the alterations in patients may be less pronounced than originally anticipated they demonstrate the need for the definition and standardization of optimal methodological parameters.

The role of conflict processing in multisensory perception: behavioural and electroencephalography evidence.

To form coherent multisensory perceptual representations, the brain must solve a causal inference problem: to decide if two sensory cues originated from the same event and should be combined, or if they came from different events and should be processed independently. According to current models of multisensory integration, during this process, the integrated (common cause) and segregated (different causes) internal perceptual models are entertained. In the present study, we propose that the causal inference process involves competition between these alternative perceptual models that engages the brain mechanisms of conflict processing. To test this hypothesis, we conducted two experiments, measuring reaction times (RTs) and electroencephalography, using an audiovisual ventriloquist illusion paradigm with varying degrees of intersensory disparities. Consistent with our hypotheses, incongruent trials led to slower RTs and higher fronto-medial theta power, both indicative of conflict. We also predicted that intermediate disparities would yield slower RTs and higher theta power when compared to congruent stimuli and to large disparities, owing to the steeper competition between causal models. Although this prediction was only validated in the RT study, both experiments displayed the anticipated trend. In conclusion, our findings suggest a potential involvement of the conflict mechanisms in multisensory integration of spatial information. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Cinematographic continuity edits across shot scales and camera angles: an ERP analysis.

Film editing has attracted great theoretical and practical interest since the beginnings of cinematography. In recent times, the neural correlates of visual transitions at edit cuts have been at the focus of attention in neurocinematics. Many Event Related Potential (ERP) studies studies have reported the consequences of cuts involving narrative discontinuities, and violations of standard montage rules. However, less is known about edits that are meant to induce continuity. Here, we addressed the neural correlates of continuity editing involving scale, and angle variations across the cut within the same scene, two of the most popular devices used for continuity editing. We recorded the electroencephalographic signal obtained from 20 viewers as they watched four different cinematographic excerpts to extract ERPs at edit points. First, we were able to reproduce the general time and scalp distribution of the typical ERPs to filmic cuts in prior studies. Second, we found significant ERP modulations triggered by scale changes (scale out, scale in, or maintaining the same scale). Edits involving an increase in scale (scale out) led to amplification of the ERP deflection, and scale reduction (scale in) led to decreases, compared to edits that kept scale across the cut. These modulations coincide with the time window of the N300 and N400 components and, according to previous findings, their amplitude has been associated with the likelihood of consciously detecting the edit. Third, we did not detect similar modulations as a function of angle variations across the cut. Based on these findings, we suggest that cuts involving reduction of scale are more likely to go unnoticed, than ones that scale out. This relationship between scale in/out and visibility is documented in film edition manuals. Specifically, in order to achieve fluidity in a scene, the edition is designed from the most opened shots to the most closed ones.

Using occipital ⍺-bursts to modulate behavior in real-time.

Pre-stimulus endogenous neural activity can influence the processing of upcoming sensory input and subsequent behavioral reactions. Despite it is known that spontaneous oscillatory activity mostly appears in stochastic bursts, typical approaches based on trial averaging fail to capture this. We aimed at relating spontaneous oscillatory bursts in the alpha band (8-13 Hz) to visual detection behavior, via an electroencephalography-based brain-computer interface (BCI) that allowed for burst-triggered stimulus presentation in real-time. According to alpha theories, we hypothesized that visual targets presented during alpha-bursts should lead to slower responses and higher miss rates, whereas targets presented in the absence of bursts (low alpha activity) should lead to faster responses and higher false alarm rates. Our findings support the role of bursts of alpha oscillations in visual perception and exemplify how real-time BCI systems can be used as a test bench for brain-behavioral theories.

Long-Range α-Synchronization as Control Signal for BCI: A Feasibility Study.

Shifts in spatial attention are associated with variations in α band (α, 8-14 Hz) activity, specifically in interhemispheric imbalance. The underlying mechanism is attributed to local α-synchronization, which regulates local inhibition of neural excitability, and frontoparietal synchronization reflecting long-range communication. The direction-specific nature of this neural correlate brings forward its potential as a control signal in brain-computer interfaces (BCIs). In the present study, we explored whether long-range α-synchronization presents lateralized patterns dependent on voluntary attention orienting and whether these neural patterns can be picked up at a single-trial level to provide a control signal for active BCI. We collected electroencephalography (EEG) data from a cohort of healthy adults (n = 10) while performing a covert visuospatial attention (CVSA) task. The data show a lateralized pattern of α-band phase coupling between frontal and parieto-occipital regions after target presentation, replicating previous findings. This pattern, however, was not evident during the cue-to-target orienting interval, the ideal time window for BCI. Furthermore, decoding the direction of attention trial-by-trial from cue-locked synchronization with support vector machines (SVMs) was at chance level. The present findings suggest EEG may not be capable of detecting long-range α-synchronization in attentional orienting on a single-trial basis and, thus, highlight the limitations of this metric as a reliable signal for BCI control.

A practical guide for studying human behavior in the lab.

In the last few decades, the field of neuroscience has witnessed major technological advances that have allowed researchers to measure and control neural activity with great detail. Yet, behavioral experiments in humans remain an essential approach to investigate the mysteries of the mind. Their relatively modest technological and economic requisites make behavioral research an attractive and accessible experimental avenue for neuroscientists with very diverse backgrounds. However, like any experimental enterprise, it has its own inherent challenges that may pose practical hurdles, especially to less experienced behavioral researchers. Here, we aim at providing a practical guide for a steady walk through the workflow of a typical behavioral experiment with human subjects. This primer concerns the design of an experimental protocol, research ethics, and subject care, as well as best practices for data collection, analysis, and sharing. The goal is to provide clear instructions for both beginners and experienced researchers from diverse backgrounds in planning behavioral experiments.

Balance between breadth and depth in human many-alternative decisions.

Many everyday life decisions require allocating finite resources, such as attention or time, to examine multiple available options, like choosing a food supplier online. In cases like these, resources can be spread across many options (breadth) or focused on a few of them (depth). Whilst theoretical work has described how finite resources should be allocated to maximize utility in these problems, evidence about how humans balance breadth and depth is currently lacking. We introduce a novel experimental paradigm where humans make a many-alternative decision under finite resources. In an imaginary scenario, participants allocate a finite budget to sample amongst multiple apricot suppliers in order to estimate the quality of their fruits, and ultimately choose the best one. We found that at low budget capacity participants sample as many suppliers as possible, and thus prefer breadth, whereas at high capacities participants sample just a few chosen alternatives in depth, and intentionally ignore the rest. The number of alternatives sampled increases with capacity following a power law with an exponent close to 3/4. In richer environments, where good outcomes are more likely, humans further favour depth. Participants deviate from optimality and tend to allocate capacity amongst the selected alternatives more homogeneously than it would be optimal, but the impact on the outcome is small. Overall, our results undercover a rich phenomenology of close-to-optimal behaviour and biases in complex choices.

The influence of temporal unpredictability on the electrophysiological mechanisms of neural entrainment.

Neural entrainment, or the synchronization of endogenous oscillations to exogenous rhythmic events, has been postulated as a powerful mechanism underlying stimulus prediction. Nevertheless, studies that have explored the benefits of neural entrainment on attention, perception, and other cognitive functions have received criticism, which could compromise their theoretical and clinical value. Therefore, the aim of the present study was [1] to confirm the presence of entrainment using a set of pre-established criteria and [2] to establish whether the reported behavioral benefits of entrainment remain when temporal predictability related to target appearance is reduced. To address these points, we adapted a previous neural entrainment paradigm to include: a variable entrainer length and increased target-absent trials, and instructing participants to respond only if they had detected a target, to avoid guessing. Thirty-six right-handed women took part in this study. Our results indicated a significant alignment of neural activity to the external periodicity as well as a persistence of phase alignment beyond the offset of the driving signal. This would appear to indicate that neural entrainment triggers preexisting endogenous oscillations, which cannot simply be explained as a succession of event-related potentials associated with the stimuli, expectation and/or motor response. However, we found no behavioral benefit for targets in-phase with entrainers, which would suggest that the effect of neural entrainment on overt behavior may be more limited than expected. These results help to clarify the mechanistic processes underlying neural entrainment and provide new insights on its applications.

Alpha fluctuations regulate the accrual of visual information to awareness.

Endogenous brain processes play a paramount role in shaping up perceptual phenomenology. This is illustrated by the alternations experienced by humans (and other animals) when watching perceptually ambiguous, static images. We hypothesised that endogenous alpha fluctuations in the visual cortex pace the accumulation of sensory information leading to perceptual outcomes. Here, we addressed this hypothesis using binocular rivalry combined with visual entrainment and electroencephalography in humans (64 female, 53 male). The results revealed a correlation between the individual frequency of alpha oscillations in the occipital cortex and perceptual alternation rates experienced during binocular rivalry. In subsequent experiments we show that regulating endogenous brain activity via rhythmic entrainment produced corresponding changes in perceptual alternation rate. These changes were observed only in the alpha range but not at lower entrainment frequencies, and were much reduced when using arrhythmic stimulation. Additionally, entraining at frequencies above the alpha range did not result in speeding up perceptual alternation rates. Overall, these findings support the notion that visual information is accumulated via alpha cycles to promote the emergence of conscious perceptual representations. We suggest that models of binocular rivalry incorporating posterior alpha as a pacemaker can provide an important advance in the comprehension of the dynamics of visual awareness.

Conflict monitoring and attentional adjustment during binocular rivalry.

To make sense of ambiguous and, at times, fragmentary sensory input, the brain must rely on a process of active interpretation. At any given moment, only one of several possible perceptual representations prevails in our conscious experience. Our hypothesis is that the competition between alternative representations induces a pattern of neural activation resembling cognitive conflict, eventually leading to fluctuations between different perceptual outcomes in the case of steep competition. To test this hypothesis, we probed changes in perceptual awareness between competing images using binocular rivalry. We drew our predictions from the conflict monitoring theory, which holds that cognitive control is invoked by the detection of conflict during information processing. Our results show that fronto-medial theta oscillations (5-7 Hz), an established electroencephalography (EEG) marker of conflict, increases right before perceptual alternations and decreases thereafter, suggesting that conflict monitoring occurs during perceptual competition. Furthermore, to investigate conflict resolution via attentional engagement, we looked for a neural marker of perceptual switches as by parieto-occipital alpha oscillations (8-12 Hz). The power of parieto-occipital alpha displayed an inverse pattern to that of fronto-medial theta, reflecting periods of high interocular inhibition during stable perception, and low inhibition around moments of perceptual change. Our findings aim to elucidate the relationship between conflict monitoring mechanisms and perceptual awareness.

Can the occipital alpha-phase speed up visual detection through a real-time EEG-based brain-computer interface (BCI)?

Electrical brain oscillations reflect fluctuations in neural excitability. Fluctuations in the alpha band (α, 8-12 Hz) in the occipito-parietal cortex are thought to regulate sensory responses, leading to cyclic variations in visual perception. Inspired by this theory, some past and recent studies have addressed the relationship between α-phase from extra-cranial EEG and behavioural responses to visual stimuli in humans. The latest studies have used offline approaches to confirm α-gated cyclic patterns. However, a particularly relevant implication is the possibility to use this principle online, whereby stimuli are time-locked to specific α-phases leading to predictable outcomes in performance. Here, we aimed at providing a proof of concept for such real-time neurotechnology. Participants performed a speeded response task to visual targets that were presented upon a real-time estimation of the α-phase via an EEG closed-loop brain-computer interface (BCI). According to the theory, we predicted a modulation of reaction times (RTs) along the α-cycle. Our BCI system achieved reliable trial-to-trial phase locking of stimuli to the phase of individual occipito-parietal α-oscillations. Yet, the behavioural results did not support a consistent relation between RTs and the phase of the α-cycle neither at group nor at single participant levels. We must conclude that although the α-phase might play a role in perceptual decisions from a theoretical perspective, its impact on EEG-based BCI application appears negligible.

The dynamics of decision-making and action during active sampling.

Embodied Cognition Theories (ECTs) of decision-making propose that the decision process pervades the execution of choice actions and manifests itself in these actions. Decision-making scenarios where actions not only express the choice but also help sample information can provide a valuable, ecologically relevant model for this framework. We present a study to address this paradigmatic situation in humans. Subjects categorized (2AFC task) a central object image, blurred to different extents, by moving a cursor toward the left or right of the display. Upward cursor movements reduced the image blur and could be used to sample information. Thus, actions for decision and actions for sampling were orthogonal to each other. We analyzed response trajectories to test whether information-sampling movements co-occurred with the ongoing decision process. Trajectories were bimodally distributed, with one kind being direct towards one response option (non-sampling), and the other kind containing an initial upward component before veering off towards an option (sampling). This implies that there was an initial decision at the early stage of a trial, whether to sample information or not. Importantly, in sampling trials trajectories were not purely upward, but rather had a significant horizontal deviation early on. This result suggests that movements to sample information exhibit an online interaction with the decision process, therefore supporting the prediction of the ECTs under ecologically relevant constrains.

The phase of Theta oscillations modulates successful memory formation at encoding.

Several studies have shown that attention and perception can depend upon the phase of ongoing neural oscillations at stimulus onset. Here, we extend this idea to the memory domain. We tested the hypothesis that ongoing fluctuations in neural activity impact memory encoding in two experiments using a picture paired-associates task in order to gauge episodic memory performance. Experiment 1 was behavioural only and capitalized on the principle of phase resetting. We tested if subsequent memory performance fluctuates rhythmically, time-locked to a resetting cue presented before the to-be-remembered pairs at different time intervals. We found an indication that behavioural performance was periodically and selectively modulated at Theta frequency (~4 Hz). In Experiment 2, we focused on pre-stimulus ongoing activity using scalp EEG while participants performed a paired-associates task. The pre-registered analysis, using large electrode clusters and generic Theta and Alpha spectral ranges, returned null results of the pre-stimulus phase-behaviour correlation. However, as expected from prior literature, we found that variations in stimulus-related Theta-power predicted subsequent memory performance. Therefore, we used this post-stimulus effect in Theta power to guide a post-hoc pre-stimulus phase analysis in terms of scalp and frequency of interest. This analysis returned a correlation between the pre-stimulus Theta phase and subsequent memory. Altogether, these results suggest that pre-stimulus Theta activity at encoding may impact later memory performance.

Integrating when and what information in the left parietal lobe allows language rule generalization.

A crucial aspect when learning a language is discovering the rules that govern how words are combined in order to convey meanings. Because rules are characterized by sequential co-occurrences between elements (e.g., "These cupcakes are unbelievable"), tracking the statistical relationships between these elements is fundamental. However, purely bottom-up statistical learning alone cannot fully account for the ability to create abstract rule representations that can be generalized, a paramount requirement of linguistic rules. Here, we provide evidence that, after the statistical relations between words have been extracted, the engagement of goal-directed attention is key to enable rule generalization. Incidental learning performance during a rule-learning task on an artificial language revealed a progressive shift from statistical learning to goal-directed attention. In addition, and consistent with the recruitment of attention, functional MRI (fMRI) analyses of late learning stages showed left parietal activity within a broad bilateral dorsal frontoparietal network. Critically, repetitive transcranial magnetic stimulation (rTMS) on participants' peak of activation within the left parietal cortex impaired their ability to generalize learned rules to a structurally analogous new language. No stimulation or rTMS on a nonrelevant brain region did not have the same interfering effect on generalization. Performance on an additional attentional task showed that this rTMS on the parietal site hindered participants' ability to integrate "what" (stimulus identity) and "when" (stimulus timing) information about an expected target. The present findings suggest that learning rules from speech is a two-stage process: following statistical learning, goal-directed attention-involving left parietal regions-integrates "what" and "when" stimulus information to facilitate rapid rule generalization.

From cognitive control to visual incongruity: Conflict detection in surrealistic images.

This study explored brain responses to images that exploit incongruity as a creative technique, often used in advertising (i.e., surrealistic images). We hypothesized that these images would reveal responses akin to cognitive conflict resulting from incongruent trials in typical laboratory tasks (i.e., Stroop Task). Indeed, in many surrealistic images, common visual elements are juxtaposed to create un-ordinary associations with semantically conflicting representations. We expected that these images engage the conflict processing network that has been described in cognitive neuroscience theories. We addressed this hypothesis by measuring the power of mid-frontal Theta oscillations using EEG while participants watched images through a social media-like interface. Incongruent images, compared to controls, produced a significant Theta power increase, as predicted from the cognitive conflict theory. We also found increased memory for incongruent images one week after exposure, compared to the controls. These findings provide evidence for the incongruent images to effectively engage the viewer's cognitive control and boost memorability. The results of this study provide validation of cognitive theories in real-life scenarios (i.e., surrealistic ads or art) and offer insights regarding the use of neural correlates as effectiveness metrics in advertising.

Characteristic Sounds Facilitate Object Search in Real-Life Scenes.

Real-world events do not only provide temporally and spatially correlated information across the senses, but also semantic correspondences about object identity. Prior research has shown that object sounds can enhance detection, identification, and search performance of semantically consistent visual targets. However, these effects are always demonstrated in simple and stereotyped displays that lack ecological validity. In order to address identity-based cross-modal relationships in real-world scenarios, we designed a visual search task using complex, dynamic scenes. Participants searched for objects in video clips recorded from real-life scenes. Auditory cues, embedded in the background sounds, could be target-consistent, distracter-consistent, neutral, or just absent. We found that, in these naturalistic scenes, characteristic sounds improve visual search for task-relevant objects but fail to increase the salience of irrelevant distracters. Our findings generalize previous results on object-based cross-modal interactions with simple stimuli and shed light upon how audio-visual semantically congruent relationships play out in real-life contexts.

Visual Search for People Among People.

Humans can effectively search visual scenes by spatial location, visual feature, or whole object. Here, we showed that visual search can also benefit from fast appraisal of relations between individuals in human groups. Healthy adults searched for a facing (seemingly interacting) body dyad among nonfacing dyads or a nonfacing dyad among facing dyads. We varied the task parameters to emphasize processing of targets or distractors. Facing-dyad targets were more likely to recruit attention than nonfacing-dyad targets (Experiments 1, 2, and 4). Facing-dyad distractors were checked and rejected more efficiently than nonfacing-dyad distractors (Experiment 3). Moreover, search for an individual body was more difficult when it was embedded in a facing dyad than in a nonfacing dyad (Experiment 5). We propose that fast grouping of interacting bodies in one attentional unit is the mechanism that accounts for efficient processing of dyads within human groups and for the inefficient access to individual parts within a dyad.

The relevance of alpha phase in human perception.

Neural oscillations in the low frequencies, roughly in the alpha band (α, 5-15 Hz), have been suggested to act as a gateway from sensation to perception. This hypothesis assumes discrete perception. In particular, the amplitude and the phase of the low frequency rhythm entails a cortical mechanism which paces the access of sensory information into the cognitive system. Evidence supporting this idea includes correlations between the phase of neural oscillations and behavioral performance in perception, spatial attention and working memory. Despite a widespread confidence in the theory, these findings have been mostly based on a varied range of exploratory approaches and inferential group statistics. Here, we aimed at validating the involvement of low frequency cortical rhythm in perception and at providing a clear-cut EEG analysis pipeline. Such an analytical pipeline should support the adoption of a hypothesis-driven framework for future replications and applications. The design, the analyses and the statistical power of the present experiment were based on prior studies in which phase opposition was successfully found. However, our results provide evidence for the involvement of pre-stimulus oscillatory alpha amplitude but not phase in perception. We discuss the null findings from the present study within the existing literature.